Ratio Edit Tolerance Development Using Variations of Exploratory Data Analysis (eda) Resistant Fences Methods

1. Introduction Many data items collected by the Bureau of the Census Economic Programs are subjected to ratio edits. In a ratio edit, the ratio of two correlated items is compared to upper and lower bounds, known as tolerances. Reported items that fall outside of the tolerances are considered edit failures, and one or both of the items in an edit-failing ratio are either imputed or flagged for analyst review. The efficiency of the ratio edit is consequently dependent on the selected tolerances. In 1996, Thompson and Sigman conducted research to determine a statistical method of automatically setting tolerance limits that works well for different sets of economic data for use in the 1997 economic census. We evaluated these methods on two sets of historical data: the 1994 Annual Survey of Manufactures (ASM) and the 1992 Business Census. In both data sets, we achieved success with some variations of an Exploratory Data Analysis (EDA) method called resistant fences. The resistant fences rules flag a ratio as an outlier when it is k interquartile ranges outside of the first or third quartiles (k is a constant). In the Business Census applications, the resistant fences methods worked best when the original distributions of ratios were symmetrized using a power transformation before applying the resistant fences methods (final tolerances were obtained using the inverse-transform on the initial limits). However, in other data sets, the symmetrizing effort has not proved worthwhile. Lanska and Kryscio (1997) propose a variation of the resistant fences rules for asymmetric distributions: use the distance between the first quartile and the median and the distance between the third quartile and the median for the upper and lower fences instead of the interquartile range, thus elongating the fences in the direction of the skewness of the distribution. This paper investigates this method for tolerance development, in comparison with the previously described resistant methods on several sets of simulated data. In Section 2, I describe the resistant methods investigated for tolerance development. Section 3 describes the simulated data. Section 4 presents an evaluation of these methods. Section 5 presents my recommendation. 2. Resistant Methods Used for Tolerance Development Given an ordered distribution of ratios, let q 25 = the first quartile, q 75 = the third quartile, m= the sample median, and H = (q 75-q 25), the interquartile range. Then, Resistant Fences flag outliers as ratios less than q 25-k×H or …